|
|
|
| Attribute Reduction Method Based on MapReduce-Based Improved Discrete Glowworm Swarm Algorithm and Multi-fractal Dimension |
| LU Yujia1,2, NI Zhiwei1,2, ZHU Xuhui1,2, XU Lifen1,2, WU Zhangjun1,2 |
1.School of Management, Hefei University of Technology, Hefei 230009
2.Key Laboratory of Process Optimization and Intelligent Decision-Making of Ministry of Education, Hefei University of Technology, Hefei 230009 |
|
|
|
|
Abstract To solve the problem of attribute reduction in a big data environment, an attribute reduction method based on MapReduce-based improved discrete glowworm swarm algorithm(IDGSO) and multi-fractal dimension(MFD) is proposed. Firstly, the moving way of glowworm individuals is discretized to avoid the algorithm falling into local optimum, and the migration strategy and Gaussian mutation strategy are introduced. An improved discrete glowworm swarm algorithm is proposed. Secondly, the improved discrete glowworm algorithm combined with multi-fractal dimension is applied to attribute reduction. Finally, to solve the problem mentioned above, the MapReduce programming model is adopted to realize the parallelization of IDGSO and MFD. Experiments on UCI datasets and the real meteorological datasets show that the proposed method produces high efficiency, effectiveness and feasibility of reduction.
|
|
Received: 30 January 2018
|
|
|
| Fund:Supported by Training Program of Major Research Plan of National Natural Science Foundation of China(No. 91546108), Major Projects of National Natural Science Foundation of China(No.91490725), Innovation Research Group Project of National Natural Science Foundation of China(No.71521001), Anhui Provincial Natural Science Foundation(No.1708085MG169), The Humanities and Social Science Research Project of the Anhui Provincial Education Department(No.JS2017AJRW0135) |
| About author:: (LU Yujia, master student. Her research interests include swarm intelligence algorithm, data mining and machine learning.)(NI Zhiwei(corresponding author), Ph.D., professor. His research interests include artificial intelligence, machine learning and cloud computing.)(ZHU Xuhui, Ph.D. candidate. His research interests include evolutionary computing and machine learning.)(XU Lifen, master student. Her research interests include attribute reduction and data preprocessing.)(WU Zhangjun, Ph.D., associate profe-ssor. His research interests include intelligent algorithm and process management.) |
|
|
|
[1] MAJHI S K, SHIAL G. Challenges in Big Data Cloud Computing and Future Research Prospects: A Review. Smart Computing Review, 2015: 5(4): 340-345.
[2] BECKWITH R. Managing Big Data: Cloud Computing and Co-location Centers. Journal of Petroleum Technology, 2011, 63(10): 42-45.
[3] 常犁云,王国胤,吴 渝.一种基于Rough Set理论的属性约简及规则提取方法.软件学报, 1999, 10(11): 1206-1211.
(CHANG L Y, WANG G Y, WU Y. An Approach for Attribute Reduction and Rule Generation Based on Rough Set Theory. Journal of Software, 1999, 10(11): 1206-1211.)
[4] 叶东毅,廖建坤.基于二进制粒子群优化的一个最小属性约简算法.模式识别与人工智能, 2007, 20(3): 295-300.
(YE D Y, LIAO J K. Minimum Attribute Reduction Algorithm Based on Binary Particle Swarm Optimization. Pattern Recognition and Artificial Intelligence, 2007, 20(3): 295-300.)
[5] LIU H, YU L. Toward Integrating Feature Selection Algorithms for Classification and Clustering. IEEE Transactions on Knowledge and Data Engineering, 2005, 17(4): 491-502.
[6] 宣国荣,柴佩琪.基于巴氏距离的特征选择.模式识别与人工智能, 1996, 9(4): 324-329.
(XUAN G R, CHAI P Q. Feature Selection Based on Barren Distance. Pattern Recognition and Artificial Intelligence, 1996, 9(4): 324-329.)
[7] HUANG D, CHOW T W S. Effective Feature Selection Scheme Using Mutual Information. Neurocomputing, 2005, 63: 325-343.
[8] WANG X Y, YANG J, TENG X L, et al. Feature Selection Based on Rough Sets and Particle Swarm Optimization. Pattern Recognition Letters, 2007, 28(4): 459-471.
[9] CHEN Y M, ZHU Q X, XU H R. Finding Rough Set Reducts with Fish Swarm Algorithm. Knowledge-Based Systems, 2015, 81: 22-29.
[10] LUAN X Y, LI Z P, LIU T Z. A Novel Attribute Reduction Algorithm Based on Rough Set and Improved Artificial Fish Swarm Algorithm. Neurocomputing, 2016, 174: 522-529.
[11] JR TRAINA C, TRAINA A, WU L, et al. Fast Feature Selection Using Fractal Dimension. Journal of Information and Data Management, 2010, 1(1): 3-16.
[12] CAMASTRA F, VINCIARELLI A. Estimating the Intrinsic Dimension of Data with a Fractal-Based Method. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2002, 24(10): 1404-1407.
[13] 倪志伟,肖宏旺,伍章俊,等.基于改进离散型萤火虫群优化算法和分形维数的属性选择方法.模式识别与人工智能, 2013, 26(12): 1170-1178.
(NI Z W, XIAO H W, WU Z J, et al. Attribute Selection Method Based on Improved Discrete Glowworm Swarm Optimization and Fractal Dimension. Pattern Recognition and Artificial Intelligence, 2013, 26(12): 1169-1178.)
[14] 李敬明,倪志伟,许 莹,等.基于二进制萤火虫算法的属性选择方法研究.系统科学与数学, 2017, 37(2): 407-424.
(LI J M, NI Z W, XU Y, et al. Research on Attribute Selection Method Based on Binary Glowworm Swarm Optimization Algorithm. Journal of Systems Science and Mathematical Sciences, 2017, 37(2): 407-424.)
[15] ZHANG C, NI Z W, NI L P, et al. Feature Selection Method Based on Multi-fractal Dimension and Harmony Search Algorithm and Its Application. International Journal of Systems Science, 2016, 47(14): 3476-3486.
[16] 朱旭辉,倪志伟,程美英,等.融合协同进化离散型人工鱼群算法和多重分形的雾霾预测方法.系统工程理论与实践, 2017, 37(4): 999-1010.
(ZHU X H, NI Z W, CHENG M Y, et al. Haze Prediction Me-thod Based on Multi-fractal Dimension and Co-evolution Discrete Artificial Fish Swarm Algorithm. Systems Engineering-Theory & Practice, 2017, 37(4): 999-1010.)
[17] 马 昕,林丽清.蚁群算法在面向属性的数据约简中的应用.计算机仿真, 2007, 24(9): 158-160.
(MA X, LIN L Q. Application Ant Colony Algorithm in Attribute-Oriented Data Reduction. Computer Simulation, 2007, 24(9): 158-160.)
[18] 李订芳,章 文,李贵斌,等.基于可行域的遗传约简算法.小型微型计算机系统, 2006, 27(2): 312-315.
(LI D F, ZHANG W, LI G B, et al. Genetic Reduction Algorithm Based on Feasible Region. Journal of Chinese Mini-Micro Compu-ter Systems, 2006, 27(2): 312-315.)
[19] SUGUNA N, THANUSHKODI K. A Novel Rough Set Reduct Algorithm for Medical Domain Based on Bee Colony Optimization. Journal of Computing, 2010, 2(6): 49-54.
[20] KRISHNANAND K N, GHOSE D. Glowworm Swarm Based Optimization Algorithm for Multimodal Functions with Collective Robotics Applications. Multiagent and Grid Systems, 2006, 2(3): 209-222.
[21] 程美英,倪志伟,朱旭辉.基于生命周期的二元蚁群优化算法.模式识别与人工智能, 2014, 27(11): 1005-1014.
(CHENG M Y, NI Z W, ZHU X H. Lifecycle-Based Binary Ant Colony Optimization Algorithm. Pattern Recognition and Artificial Intelligence, 2014, 27(11): 1005-1014.)
[22] 莫愿斌,刘付永,张宇楠.带高斯变异的人工萤火虫优化算法.计算机应用研究, 2013, 30(1): 121-123.
(MO Y B, LIU F Y, ZHANG Y N. Artificial Glowworm Swarm Optimization Algorithm with Gauss Mutation. Application Research of Computers, 2013, 30(1): 121-123.)
[23] 罗雪晖,杨 烨,李 霞.改进混合蛙跳算法求解旅行商问题.通信学报, 2009, 30(7): 130-135.
(LUO X H, YANG Y, LI X. Modified Shuffled Frog-Leaping Algorithm to Solve Traveling Salesman Problem. Journal on Communications, 2009, 30(7): 130-135.) |
|
|
|
2018, Vol. 31 
